[AutoParallel] Enhance processmesh #72052
Conversation
|
你的PR提交成功，感谢你对开源项目的贡献! |
xuxinyi389
force-pushed
the
enhance_processmesh
branch
from
f6b4054 to
b589534
Compare
|
Sorry to inform you that b589534's CIs have passed for more than 7 days. To prevent PR conflicts, you need to re-run all CIs manually. |
xuxinyi389
changed the title
xuxinyi389
force-pushed
the
enhance_processmesh
branch
from
fd92afc to
38f9e95
Compare
|
/re-run approval |
| return ProcessMesh(new_mesh, new_dim_names) | ||
|
|
||
| def get_submesh_with_dim( |
There was a problem hiding this comment.
这个方法和get_mesh_with_dim有什么区别？
There was a problem hiding this comment.
get_mesh_with_dim 只是对 mesh 的一个简单重排，如mesh.get_mesh_with_dim(“dp”)只是把mesh的dp维放在最外维，并没有减少mesh内process_ids。mesh.get_submesh_with_dim("dp")则是获取包含当前rank的dp通信组的submesh。比如说：mesh_2d = dist.ProcessMesh([[0, 1, 2, 3], [4, 5, 6, 7]], dim_names=["dp", "tp"])
dp_mesh = mesh_2d.get_submesh_with_dim("dp")
on rank 0, 4 returns a 1D submesh of ProcessMesh:([0, 4]).
on rank 1, 5 returns a 1D submesh of ProcessMesh:([1, 5]).
on rank 2, 6 returns a 1D submesh of ProcessMesh:([2, 6]).
on rank 3, 7 returns a 1D submesh of ProcessMesh:([3, 7]).
* processmesh support convert group * add_test * fix_test * fix_en_docs * move_test * fix_bugs_of_get_mesh_with_dim
PR Category
Auto Parallel
PR Types
Improvements
Description
card-73263
Example：
mesh_2d = dist.ProcessMesh([[0, 1, 2, 3], [4, 5, 6, 7]], dim_names=["dp", "tp"])
dp_mesh = mesh_2d["dp"]
tp_mesh = mesh_2d["tp"]
Calling mesh_2d["dp"] on rank 0, 4 returns a 1D submesh of DeviceMesh:([0, 4]).
Calling mesh_2d["dp"] on rank 1, 5 returns a 1D submesh of DeviceMesh:([1, 5]).
Calling mesh_2d["dp"] on rank 2, 6 returns a 1D submesh of DeviceMesh:([2, 6]).
Calling mesh_2d["dp"] on rank 3, 7 returns a 1D submesh of DeviceMesh:([3, 7]).
Calling mesh_2d["tp"] on rank 0, 1, 2, 3 returns a 1D submesh of DeviceMesh:([0, 1, 2, 3]).
Calling mesh_2d["tp"] on rank 4, 5, 6, 7 returns a 1D submesh of DeviceMesh:([4, 5, 6, 7]).